Rewrite the `ci.py` script in Rust #136864

Kobzol · 2025-02-11T12:25:12Z

It would seem that I would learn by now that any script written in Python will become unmaintainable sooner or later, but alas..

r? @marcoieni

try-job: aarch64-gnu
try-job: dist-x86_64-linux-alt
try-job: x86_64-msvc-ext2

Fixes: #137013

rustbot · 2025-02-11T12:25:23Z

The rustc-dev-guide subtree was changed. If this PR only touches the dev guide consider submitting a PR directly to rust-lang/rustc-dev-guide otherwise thank you for updating the dev guide with your changes.

cc @BoxyUwU, @jieyouxu, @Kobzol

jieyouxu

Looks very cool. Not an infra reviewer, just some drive-by comments :P

src/ci/citool/src/main.rs

marcoieni

comments on the documentation

src/ci/github-actions/jobs.yml

src/doc/rustc-dev-guide/src/building/optimized-build.md

.github/workflows/ci.yml

jieyouxu · 2025-02-11T13:33:00Z

There is a risk (arguably tiny) that may or may not be worth considering, which is that with a stable toolchain bump (through GHA's rustup I think?) that this somehow can't build anymore¹ or fail at runtime due to $reasons (which would be a bit awkward)²³. Is it somehow possible to pin which rust toolchain is used to build this in extraordinary scenarios?

I dunno if a rustup toolchain.toml override works in a subdirectory 🤷

maybe itself, or maybe one of its transitive deps ↩
for normal crates I would not have considered this, but this is the CI which kinda needs to keep working so ↩
maybe lints, so maybe building this CI crate itself should never use -Dwarnings? ↩

src/ci/citool/Cargo.toml

marcoieni · 2025-02-11T14:41:18Z

I wrote a test. Add this to Cargo.toml:

[dev-dependencies]
insta = "1"

test file:

use std::process::{Command, Stdio};

const TEST_JOBS_YML_PATH: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/tests/jobs.yml");

#[test]
fn test_cargo_check() {
    let output = Command::new("cargo")
        .args(["run", "-q", "calculate-job-matrix", "--jobs-file", TEST_JOBS_YML_PATH])
        .env("GITHUB_EVENT_NAME", "push")
        .env("COMMIT_MESSAGE", "test commit message")
        .env("GITHUB_REF", "refs/heads/auto")
        .stdout(Stdio::piped())
        .output()
        .expect("Failed to execute command");

    let stdout = String::from_utf8(output.stdout).unwrap();
    let stderr = String::from_utf8(output.stderr).unwrap();
    if !output.status.success() {
        panic!("cargo run failed: {}\n{}", stdout, stderr);
    }
    insta::assert_snapshot!(stdout, @r#"
    jobs=[{"name":"aarch64-gnu","full_name":"auto - aarch64-gnu","os":"ubuntu-22.04-arm","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","DEPLOY_BUCKET":"rust-lang-ci2","TOOLSTATE_PUBLISH":"1"},"free_disk":true},{"name":"x86_64-gnu-llvm-18-1","full_name":"auto - x86_64-gnu-llvm-18-1","os":"ubuntu-24.04","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","DEPLOY_BUCKET":"rust-lang-ci2","DOCKER_SCRIPT":"stage_2_test_set1.sh","IMAGE":"x86_64-gnu-llvm-18","READ_ONLY_SRC":"'0'","RUST_BACKTRACE":"1","TOOLSTATE_PUBLISH":"1"},"free_disk":true},{"name":"aarch64-apple","full_name":"auto - aarch64-apple","os":"macos-14","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","DEPLOY_BUCKET":"rust-lang-ci2","MACOSX_DEPLOYMENT_TARGET":"11.0","MACOSX_STD_DEPLOYMENT_TARGET":"11.0","NO_DEBUG_ASSERTIONS":"1","NO_LLVM_ASSERTIONS":"1","NO_OVERFLOW_CHECKS":"1","RUSTC_RETRY_LINKER_ON_SEGFAULT":"1","RUST_CONFIGURE_ARGS":"--enable-sanitizers --enable-profiler --set rust.jemalloc","SCRIPT":"./x.py --stage 2 test --host=aarch64-apple-darwin --target=aarch64-apple-darwin","SELECT_XCODE":"/Applications/Xcode_15.4.app","TOOLSTATE_PUBLISH":"1","USE_XCODE_CLANG":"1"}},{"name":"dist-i686-msvc","full_name":"auto - dist-i686-msvc","os":"windows-2022","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","CODEGEN_BACKENDS":"llvm,cranelift","DEPLOY_BUCKET":"rust-lang-ci2","DIST_REQUIRE_ALL_TOOLS":"1","RUST_CONFIGURE_ARGS":"--build=i686-pc-windows-msvc --host=i686-pc-windows-msvc --target=i686-pc-windows-msvc,i586-pc-windows-msvc --enable-full-tools --enable-profiler","SCRIPT":"python x.py dist bootstrap --include-default-paths","TOOLSTATE_PUBLISH":"1"}}]
    run_type=auto
    "#);
}

diffs of main.rs:

diff --git a/src/ci/citool/src/main.rs b/src/ci/citool/src/main.rs
index 9a22d1264fa..4c6591443a2 100644
--- a/src/ci/citool/src/main.rs
+++ b/src/ci/citool/src/main.rs
@@ -1,5 +1,5 @@
-use std::collections::HashMap;
-use std::path::Path;
+use std::collections::BTreeMap;
+use std::path::{Path, PathBuf};
 use std::process::Command;
 
 use anyhow::Context;
@@ -17,13 +17,13 @@ struct Job {
     name: String,
     /// GitHub runner on which the job should be executed
     os: String,
-    env: HashMap<String, Value>,
+    env: BTreeMap<String, Value>,
     /// Should the job be only executed on a specific channel?
     #[serde(default)]
     only_on_channel: Option<String>,
     /// Rest of attributes that will be passed through to GitHub actions
     #[serde(flatten)]
-    extra_keys: HashMap<String, Value>,
+    extra_keys: BTreeMap<String, Value>,
 }
 
 impl Job {
@@ -44,11 +44,11 @@ fn image(&self) -> String {
 #[derive(serde::Deserialize, Debug)]
 struct JobEnvironments {
     #[serde(rename = "pr")]
-    pr_env: HashMap<String, Value>,
+    pr_env: BTreeMap<String, Value>,
     #[serde(rename = "try")]
-    try_env: HashMap<String, Value>,
+    try_env: BTreeMap<String, Value>,
     #[serde(rename = "auto")]
-    auto_env: HashMap<String, Value>,
+    auto_env: BTreeMap<String, Value>,
 }
 
 #[derive(serde::Deserialize, Debug)]
@@ -71,7 +71,7 @@ fn find_auto_job_by_name(&self, name: &str) -> Option<Job> {
 }
 
 fn load_job_db(path: &Path) -> anyhow::Result<JobDatabase> {
-    let db = std::fs::read_to_string(path)?;
+    let db = read_to_string(path)?;
     let mut db: Value = serde_yaml::from_str(&db)?;
 
     // We need to expand merge keys (<<), because serde_yaml can't deal with them
@@ -92,9 +92,9 @@ struct GithubActionsJob {
     /// prefix (PR/try/auto).
     full_name: String,
     os: String,
-    env: HashMap<String, String>,
+    env: BTreeMap<String, String>,
     #[serde(flatten)]
-    extra_keys: HashMap<String, serde_json::Value>,
+    extra_keys: BTreeMap<String, serde_json::Value>,
 }
 
 /// Type of workflow that is being executed on CI
@@ -175,7 +175,7 @@ fn skip_jobs(jobs: Vec<Job>, channel: &str) -> Vec<Job> {
         .collect()
 }
 
-fn to_string_map(map: &HashMap<String, Value>) -> HashMap<String, String> {
+fn to_string_map(map: &BTreeMap<String, Value>) -> BTreeMap<String, String> {
     map.iter()
         .map(|(key, value)| {
             (
@@ -232,7 +232,7 @@ fn calculate_jobs(
     let jobs = jobs
         .into_iter()
         .map(|job| {
-            let mut env: HashMap<String, String> = to_string_map(base_env);
+            let mut env: BTreeMap<String, String> = to_string_map(base_env);
             env.extend(to_string_map(&job.env));
             let full_name = format!("{prefix} - {}", job.name);
 
@@ -314,7 +314,7 @@ fn run_workflow_locally(db: JobDatabase, job_type: JobType, name: String) -> any
     };
     let job = find_linux_job(&jobs, &name).with_context(|| format!("Cannot find job {name}"))?;
 
-    let mut custom_env: HashMap<String, String> = HashMap::new();
+    let mut custom_env: BTreeMap<String, String> = BTreeMap::new();
     // Replicate src/ci/scripts/setup-environment.sh
     // Adds custom environment variables to the job
     if name.starts_with("dist-") {
@@ -340,7 +340,10 @@ fn run_workflow_locally(db: JobDatabase, job_type: JobType, name: String) -> any
 enum Args {
     /// Calculate a list of jobs that should be executed on CI.
     /// Should only be used on CI inside GitHub actions.
-    CalculateJobMatrix,
+    CalculateJobMatrix {
+        #[clap(long)]
+        jobs_file: Option<PathBuf>,
+    },
     /// Execute a given CI job locally.
     #[clap(name = "run-local")]
     RunJobLocally {
@@ -362,19 +365,29 @@ enum JobType {
 
 fn main() -> anyhow::Result<()> {
     let args = Args::parse();
-    let db = load_job_db(Path::new(JOBS_YML_PATH)).context("Cannot load jobs.yml")?;
+    let default_jobs_file = Path::new(JOBS_YML_PATH);
+    let default_db = |jobs_path| load_job_db(jobs_path).context("Cannot load jobs.yml");
 
     match args {
-        Args::CalculateJobMatrix => {
+        Args::CalculateJobMatrix { jobs_file } => {
+            let jobs_path = jobs_file.as_deref().unwrap_or(default_jobs_file);
             let gh_ctx = load_github_ctx()
                 .context("Cannot load environment variables from GitHub Actions")?;
-            let channel = std::fs::read_to_string(Path::new(CI_DIRECTORY).join("channel"))
+            let channel = read_to_string(Path::new(CI_DIRECTORY).join("channel"))
                 .context("Cannot read channel file")?;
 
-            calculate_job_matrix(db, gh_ctx, &channel).context("Failed to calculate job matrix")?;
+            calculate_job_matrix(default_db(jobs_path)?, gh_ctx, &channel)
+                .context("Failed to calculate job matrix")?;
+        }
+        Args::RunJobLocally { job_type, name } => {
+            run_workflow_locally(default_db(default_jobs_file)?, job_type, name)?
         }
-        Args::RunJobLocally { job_type, name } => run_workflow_locally(db, job_type, name)?,
     }
 
     Ok(())
 }
+
+fn read_to_string<P: AsRef<Path>>(path: P) -> anyhow::Result<String> {
+    let error = format!("Cannot read file {:?}", path.as_ref());
+    std::fs::read_to_string(path).context(error)
+}

tests/jobs.yml:

runners:
  - &base-job
    env: { }

  - &job-linux-4c
    os: ubuntu-24.04
    # Free some disk space to avoid running out of space during the build.
    free_disk: true
    <<: *base-job

  - &job-linux-16c
    os: ubuntu-22.04-16core-64gb
    <<: *base-job

  - &job-macos-m1
    os: macos-14
    <<: *base-job

  - &job-windows
    os: windows-2022
    <<: *base-job

  - &job-aarch64-linux
    # Free some disk space to avoid running out of space during the build.
    free_disk: true
    os: ubuntu-22.04-arm
    <<: *base-job
envs:
  env-x86_64-apple-tests: &env-x86_64-apple-tests
    SCRIPT: ./x.py --stage 2 test --skip tests/ui --skip tests/rustdoc -- --exact
    RUST_CONFIGURE_ARGS: --build=x86_64-apple-darwin --enable-sanitizers --enable-profiler --set rust.jemalloc
    RUSTC_RETRY_LINKER_ON_SEGFAULT: 1
    # Ensure that host tooling is tested on our minimum supported macOS version.
    MACOSX_DEPLOYMENT_TARGET: 10.12
    MACOSX_STD_DEPLOYMENT_TARGET: 10.12
    SELECT_XCODE: /Applications/Xcode_15.2.app
    NO_LLVM_ASSERTIONS: 1
    NO_DEBUG_ASSERTIONS: 1
    NO_OVERFLOW_CHECKS: 1

  production:
    &production
    DEPLOY_BUCKET: rust-lang-ci2
    # AWS_SECRET_ACCESS_KEYs are stored in GitHub's secrets storage, named
    # AWS_SECRET_ACCESS_KEY_<keyid>. Including the key id in the name allows to
    # rotate them in a single branch while keeping the old key in another
    # branch, which wouldn't be possible if the key was named with the kind
    # (caches, artifacts...).
    CACHES_AWS_ACCESS_KEY_ID: AKIA46X5W6CZI5DHEBFL
    ARTIFACTS_AWS_ACCESS_KEY_ID: AKIA46X5W6CZN24CBO55
    AWS_REGION: us-west-1
    TOOLSTATE_PUBLISH: 1

  try:
    <<: *production
    # The following env var activates faster `try` builds in `opt-dist` by, e.g.
    # - building only the more commonly useful components (we rarely need e.g. rust-docs in try
    #   builds)
    # - not running `opt-dist`'s post-optimization smoke tests on the resulting toolchain
    #
    # If you *want* these to happen however, temporarily comment it before triggering a try build.
    DIST_TRY_BUILD: 1

  auto:
    <<: *production

  pr:
    PR_CI_JOB: 1

# Jobs that run on each push to a pull request (PR)
# These jobs automatically inherit envs.pr, to avoid repeating
# it in each job definition.
pr:
  - name: mingw-check
    <<: *job-linux-4c
  - name: mingw-check-tidy
    continue_on_error: true
    <<: *job-linux-4c

# Jobs that run when you perform a try build (@bors try)
# These jobs automatically inherit envs.try, to avoid repeating
# it in each job definition.
try:
  - name: dist-x86_64-linux
    env:
      CODEGEN_BACKENDS: llvm,cranelift
    <<: *job-linux-16c

# Main CI jobs that have to be green to merge a commit into master
# These jobs automatically inherit envs.auto, to avoid repeating
# it in each job definition.
auto:
  - name: aarch64-gnu
    <<: *job-aarch64-linux

  # The x86_64-gnu-llvm-18 job is split into multiple jobs to run tests in parallel.
  # x86_64-gnu-llvm-18-1 skips tests that run in x86_64-gnu-llvm-18-{2,3}.
  - name: x86_64-gnu-llvm-18-1
    env:
      RUST_BACKTRACE: 1
      READ_ONLY_SRC: "0"
      IMAGE: x86_64-gnu-llvm-18
      DOCKER_SCRIPT: stage_2_test_set1.sh
    <<: *job-linux-4c


  ####################
  #  macOS Builders  #
  ####################

  - name: aarch64-apple
    env:
      SCRIPT: ./x.py --stage 2 test --host=aarch64-apple-darwin --target=aarch64-apple-darwin
      RUST_CONFIGURE_ARGS: >-
        --enable-sanitizers
        --enable-profiler
        --set rust.jemalloc
      RUSTC_RETRY_LINKER_ON_SEGFAULT: 1
      SELECT_XCODE: /Applications/Xcode_15.4.app
      USE_XCODE_CLANG: 1
      # Aarch64 tooling only needs to support macOS 11.0 and up as nothing else
      # supports the hardware, so only need to test it there.
      MACOSX_DEPLOYMENT_TARGET: 11.0
      MACOSX_STD_DEPLOYMENT_TARGET: 11.0
      NO_LLVM_ASSERTIONS: 1
      NO_DEBUG_ASSERTIONS: 1
      NO_OVERFLOW_CHECKS: 1
    <<: *job-macos-m1

  ######################
  #  Windows Builders  #
  ######################

  - name: dist-i686-msvc
    env:
      RUST_CONFIGURE_ARGS: >-
        --build=i686-pc-windows-msvc
        --host=i686-pc-windows-msvc
        --target=i686-pc-windows-msvc,i586-pc-windows-msvc
        --enable-full-tools
        --enable-profiler
      SCRIPT: python x.py dist bootstrap --include-default-paths
      DIST_REQUIRE_ALL_TOOLS: 1
      CODEGEN_BACKENDS: llvm,cranelift
    <<: *job-windows

What do you think?

Kobzol · 2025-02-11T14:56:39Z

There is a risk (arguably tiny) that may or may not be worth considering, which is that with a stable toolchain bump (through GHA's rustup I think?) that this somehow can't build anymore1 or fail at runtime due to $reasons (which would be a bit awkward)2 3. Is it somehow possible to pin which rust toolchain is used to build this in extraordinary scenarios?

It should be possible to pin the toolchain, but it would also mean that we'd need to download a new Rust toolchain on every workflow execution. Not a big deal, but still would be nice to avoid I guess.

The same situation held also for the previous Python script, updates to the Ubuntu base image could have broken it. But we manually control which runner image we use, so unless you consider Rust to be severely less stable than Python in this regard, I think it should be fine. It will be a bit more brittle because dependencies will have to be downloaded from crates.io, which can fail sometimes, but if crates.io fails, the rest of the CI workflow would fail anyway..

src/ci/citool/src/main.rs

Kobzol · 2025-02-11T15:34:53Z

Great point with BTreeMap, we should strive for determinism here. And also a good idea with the tests, they already helped me find a bug 😆

jieyouxu · 2025-02-11T15:42:58Z

The same situation held also for the previous Python script, updates to the Ubuntu base image could have broken it. But we manually control which runner image we use, so unless you consider Rust to be severely less stable than Python in this regard, I think it should be fine. It will be a bit more brittle because dependencies will have to be downloaded from crates.io, which can fail sometimes, but if crates.io fails, the rest of the CI workflow would fail anyway..

Of course, I'm not saying the python version didn't have this issue :P

marcoieni · 2025-02-11T16:09:28Z

LGTM. You can r=me when you feel it's ready 👍
It might be nice to check if the code is formatted and tests pass in CI.

Kobzol · 2025-02-11T16:30:35Z

Thanks! I added running tests on CI. I'll do a few tests with try builds.

@bors try

bors · 2025-02-11T16:31:58Z

⌛ Trying commit 39d61bc with merge e07f7e9...

Rewrite the `ci.py` script in Rust It would seem that I would learn by now that any script written in Python will become unmaintainable sooner or later, but alas.. r? `@marcoieni`

bors · 2025-02-11T16:36:32Z

💔 Test failed - checks-actions

Kobzol · 2025-02-11T16:36:43Z

@bors try

bors · 2025-02-11T16:37:53Z

⌛ Trying commit 39d61bc with merge 372fac4...

Rewrite the `ci.py` script in Rust It would seem that I would learn by now that any script written in Python will become unmaintainable sooner or later, but alas.. r? `@marcoieni` try-job: aarch64-gnu try-job: dist-x86_64-linux-alt

bors · 2025-02-11T16:41:43Z

💔 Test failed - checks-actions

Kobzol · 2025-02-17T11:28:31Z

Fixed handling of environment variables and reading of the src/ci/channel file. I tested locally that the Rust version should now return the exact same output as the Python version for PR/try/auto builds.

@bors try

bors · 2025-02-17T11:29:42Z

⌛ Trying commit 6a41c3f with merge 29559ee...

Rewrite the `ci.py` script in Rust It would seem that I would learn by now that any script written in Python will become unmaintainable sooner or later, but alas.. r? `@marcoieni` try-job: aarch64-gnu try-job: dist-x86_64-linux-alt try-job: x86_64-msvc-ext2 Fixes: rust-lang#137013

To avoid including a newline at the end, which broke `only_on_channel` comparison.

rust-log-analyzer · 2025-02-17T11:30:29Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

failures:

---- pr_jobs stdout ----
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Snapshot: pr_jobs
Source: tests/jobs.rs:42
Expression: stdout
────────────────────────────────────────────────────────────────────────────────
-old snapshot
+new results
+new results
────────────┬───────────────────────────────────────────────────────────────────
    0       │-jobs=[{"name":"mingw-check","full_name":"PR - mingw-check","os":"ubuntu-24.04","env":{"PR_CI_JOB":"1"},"free_disk":true},{"name":"mingw-check-tidy","full_name":"PR - mingw-check-tidy","os":"ubuntu-24.04","env":{"PR_CI_JOB":"1"},"continue_on_error":true,"free_disk":true}]
          0 │+jobs=[{"name":"mingw-check","full_name":"PR - mingw-check","os":"ubuntu-24.04","env":{"PR_CI_JOB":1},"free_disk":true},{"name":"mingw-check-tidy","full_name":"PR - mingw-check-tidy","os":"ubuntu-24.04","env":{"PR_CI_JOB":1},"continue_on_error":true,"free_disk":true}]
    1     1 │ run_type=pr
error: test failed, to rerun pass `--test jobs`
────────────┴───────────────────────────────────────────────────────────────────
Stopped on the first failure. Run `cargo insta test` to run all snapshots.
thread 'pr_jobs' panicked at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/insta-1.42.1/src/runtime.rs:679:13:
snapshot assertion for 'pr_jobs' failed in line 42

---- try_jobs stdout ----
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Snapshot: try_jobs
Snapshot: try_jobs
Source: tests/jobs.rs:17
────────────────────────────────────────────────────────────────────────────────
Expression: stdout
────────────────────────────────────────────────────────────────────────────────
-old snapshot
+new results
────────────┬───────────────────────────────────────────────────────────────────
    0       │-jobs=[{"name":"dist-x86_64-linux","full_name":"try - dist-x86_64-linux","os":"ubuntu-22.04-16core-64gb","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","CODEGEN_BACKENDS":"llvm,cranelift","DEPLOY_BUCKET":"rust-lang-ci2","DIST_TRY_BUILD":"1","TOOLSTATE_PUBLISH":"1"}}]
          0 │+jobs=[{"name":"dist-x86_64-linux","full_name":"try - dist-x86_64-linux","os":"ubuntu-22.04-16core-64gb","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","CODEGEN_BACKENDS":"llvm,cranelift","DEPLOY_BUCKET":"rust-lang-ci2","DIST_TRY_BUILD":1,"TOOLSTATE_PUBLISH":1}}]
    1     1 │ run_type=try
────────────┴───────────────────────────────────────────────────────────────────
Stopped on the first failure. Run `cargo insta test` to run all snapshots.
thread 'try_jobs' panicked at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/insta-1.42.1/src/runtime.rs:679:13:
snapshot assertion for 'try_jobs' failed in line 17

---- try_custom_jobs stdout ----
Snapshot: try_custom_jobs
Source: tests/jobs.rs:33
────────────────────────────────────────────────────────────────────────────────
Expression: stdout
Expression: stdout
────────────────────────────────────────────────────────────────────────────────
-old snapshot
+new results
────────────┬───────────────────────────────────────────────────────────────────
    0       │-jobs=[{"name":"aarch64-gnu","full_name":"try - aarch64-gnu","os":"ubuntu-22.04-arm","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","DEPLOY_BUCKET":"rust-lang-ci2","DIST_TRY_BUILD":"1","TOOLSTATE_PUBLISH":"1"},"free_disk":true},{"name":"dist-i686-msvc","full_name":"try - dist-i686-msvc","os":"windows-2022","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","CODEGEN_BACKENDS":"llvm,cranelift","DEPLOY_BUCKET":"rust-lang-ci2","DIST_REQUIRE_ALL_TOOLS":"1","DIST_TRY_BUILD":"1","RUST_CONFIGURE_ARGS":"--build=i686-pc-windows-msvc --host=i686-pc-windows-msvc --target=i686-pc-windows-msvc,i586-pc-windows-msvc --enable-full-tools --enable-profiler","SCRIPT":"python x.py dist bootstrap --include-default-paths","TOOLSTATE_PUBLISH":"1"}}]
          0 │+jobs=[{"name":"aarch64-gnu","full_name":"try - aarch64-gnu","os":"ubuntu-22.04-arm","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","DEPLOY_BUCKET":"rust-lang-ci2","DIST_TRY_BUILD":1,"TOOLSTATE_PUBLISH":1},"free_disk":true},{"name":"dist-i686-msvc","full_name":"try - dist-i686-msvc","os":"windows-2022","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","CODEGEN_BACKENDS":"llvm,cranelift","DEPLOY_BUCKET":"rust-lang-ci2","DIST_REQUIRE_ALL_TOOLS":1,"DIST_TRY_BUILD":1,"RUST_CONFIGURE_ARGS":"--build=i686-pc-windows-msvc --host=i686-pc-windows-msvc --target=i686-pc-windows-msvc,i586-pc-windows-msvc --enable-full-tools --enable-profiler","SCRIPT":"python x.py dist bootstrap --include-default-paths","TOOLSTATE_PUBLISH":1}}]
    1     1 │ run_type=try
────────────┴───────────────────────────────────────────────────────────────────
Stopped on the first failure. Run `cargo insta test` to run all snapshots.
thread 'try_custom_jobs' panicked at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/insta-1.42.1/src/runtime.rs:679:13:
snapshot assertion for 'try_custom_jobs' failed in line 33

---- auto_jobs stdout ----
Snapshot: auto_jobs
Source: tests/jobs.rs:8
────────────────────────────────────────────────────────────────────────────────
Expression: stdout
Expression: stdout
────────────────────────────────────────────────────────────────────────────────
-old snapshot
+new results
────────────┬───────────────────────────────────────────────────────────────────
    0       │-jobs=[{"name":"aarch64-gnu","full_name":"auto - aarch64-gnu","os":"ubuntu-22.04-arm","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","DEPLOY_BUCKET":"rust-lang-ci2","TOOLSTATE_PUBLISH":"1"},"free_disk":true},{"name":"x86_64-gnu-llvm-18-1","full_name":"auto - x86_64-gnu-llvm-18-1","os":"ubuntu-24.04","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","DEPLOY_BUCKET":"rust-lang-ci2","DOCKER_SCRIPT":"stage_2_test_set1.sh","IMAGE":"x86_64-gnu-llvm-18","READ_ONLY_SRC":"'0'","RUST_BACKTRACE":"1","TOOLSTATE_PUBLISH":"1"},"free_disk":true},{"name":"aarch64-apple","full_name":"auto - aarch64-apple","os":"macos-14","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","DEPLOY_BUCKET":"rust-lang-ci2","MACOSX_DEPLOYMENT_TARGET":"11.0","MACOSX_STD_DEPLOYMENT_TARGET":"11.0","NO_DEBUG_ASSERTIONS":"1","NO_LLVM_ASSERTIONS":"1","NO_OVERFLOW_CHECKS":"1","RUSTC_RETRY_LINKER_ON_SEGFAULT":"1","RUST_CONFIGURE_ARGS":"--enable-sanitizers --enable-profiler --set rust.jemalloc","SCRIPT":"./x.py --stage 2 test --host=aarch64-apple-darwin --target=aarch64-apple-darwin","SELECT_XCODE":"/Applications/Xcode_15.4.app","TOOLSTATE_PUBLISH":"1","USE_XCODE_CLANG":"1"}},{"name":"dist-i686-msvc","full_name":"auto - dist-i686-msvc","os":"windows-2022","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","CODEGEN_BACKENDS":"llvm,cranelift","DEPLOY_BUCKET":"rust-lang-ci2","DIST_REQUIRE_ALL_TOOLS":"1","RUST_CONFIGURE_ARGS":"--build=i686-pc-windows-msvc --host=i686-pc-windows-msvc --target=i686-pc-windows-msvc,i586-pc-windows-msvc --enable-full-tools --enable-profiler","SCRIPT":"python x.py dist bootstrap --include-default-paths","TOOLSTATE_PUBLISH":"1"}}]
          0 │+jobs=[{"name":"aarch64-gnu","full_name":"auto - aarch64-gnu","os":"ubuntu-22.04-arm","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","DEPLOY_BUCKET":"rust-lang-ci2","TOOLSTATE_PUBLISH":1},"free_disk":true},{"name":"x86_64-gnu-llvm-18-1","full_name":"auto - x86_64-gnu-llvm-18-1","os":"ubuntu-24.04","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","DEPLOY_BUCKET":"rust-lang-ci2","DOCKER_SCRIPT":"stage_2_test_set1.sh","IMAGE":"x86_64-gnu-llvm-18","READ_ONLY_SRC":"0","RUST_BACKTRACE":1,"TOOLSTATE_PUBLISH":1},"free_disk":true},{"name":"aarch64-apple","full_name":"auto - aarch64-apple","os":"macos-14","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","DEPLOY_BUCKET":"rust-lang-ci2","MACOSX_DEPLOYMENT_TARGET":11.0,"MACOSX_STD_DEPLOYMENT_TARGET":11.0,"NO_DEBUG_ASSERTIONS":1,"NO_LLVM_ASSERTIONS":1,"NO_OVERFLOW_CHECKS":1,"RUSTC_RETRY_LINKER_ON_SEGFAULT":1,"RUST_CONFIGURE_ARGS":"--enable-sanitizers --enable-profiler --set rust.jemalloc","SCRIPT":"./x.py --stage 2 test --host=aarch64-apple-darwin --target=aarch64-apple-darwin","SELECT_XCODE":"/Applications/Xcode_15.4.app","TOOLSTATE_PUBLISH":1,"USE_XCODE_CLANG":1}},{"name":"dist-i686-msvc","full_name":"auto - dist-i686-msvc","os":"windows-2022","env":{"ARTIFACTS_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZN24CBO55","AWS_REGION":"us-west-1","CACHES_AWS_ACCESS_KEY_ID":"AKIA46X5W6CZI5DHEBFL","CODEGEN_BACKENDS":"llvm,cranelift","DEPLOY_BUCKET":"rust-lang-ci2","DIST_REQUIRE_ALL_TOOLS":1,"RUST_CONFIGURE_ARGS":"--build=i686-pc-windows-msvc --host=i686-pc-windows-msvc --target=i686-pc-windows-msvc,i586-pc-windows-msvc --enable-full-tools --enable-profiler","SCRIPT":"python x.py dist bootstrap --include-default-paths","TOOLSTATE_PUBLISH":1}}]
    1     1 │ run_type=auto
────────────┴───────────────────────────────────────────────────────────────────
Stopped on the first failure. Run `cargo insta test` to run all snapshots.
thread 'auto_jobs' panicked at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/insta-1.42.1/src/runtime.rs:679:13:
snapshot assertion for 'auto_jobs' failed in line 8

failures:
    auto_jobs
    pr_jobs

Kobzol · 2025-02-17T11:31:08Z

@bors try

bors · 2025-02-17T11:32:23Z

⌛ Trying commit 31acbd3 with merge 457e902...

Rewrite the `ci.py` script in Rust It would seem that I would learn by now that any script written in Python will become unmaintainable sooner or later, but alas.. r? `@marcoieni` try-job: aarch64-gnu try-job: dist-x86_64-linux-alt try-job: x86_64-msvc-ext2 Fixes: rust-lang#137013

bors · 2025-02-17T14:46:36Z

☀️ Try build successful - checks-actions
Build commit: 457e902 (457e90245a010094d720fde6243f234e9d25e103)

Kobzol · 2025-02-17T14:54:54Z

Ok now I'm confident that the script works as it should.

@rustbot ready

[WIP] Postprocess bootstrap metrics into GitHub summary Based on rust-lang#136864. r? `@ghost`

Kobzol · 2025-02-26T18:07:11Z

@marcoieni Once you have a bit of time, could you please recheck if the latest changes look fine, and if so, reapprove? I didn't want to r=you without you taking another look.

I have like five branches stacked on top of this 😂 So wanted to get it through the queue. Thanks!

marcoieni · 2025-02-27T08:40:57Z

@bors r+ rollup=never

bors · 2025-02-27T08:41:01Z

📌 Commit 31acbd3 has been approved by marcoieni

It is now in the queue for this repository.

matthiaskrgr · 2025-03-02T05:03:11Z

@bors p=4 (scheduling)

bors · 2025-03-02T09:18:05Z

⌛ Testing commit 31acbd3 with merge 4b696e6...

bors · 2025-03-02T12:34:40Z

☀️ Test successful - checks-actions
Approved by: marcoieni
Pushing 4b696e6 to master...

rust-timer · 2025-03-02T14:40:06Z

Finished benchmarking commit (4b696e6): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (primary -1.7%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.7%	[-1.7%, -1.7%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-1.7%	[-1.7%, -1.7%]	1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 773.253s -> 774.29s (0.13%)
Artifact size: 361.91 MiB -> 361.94 MiB (0.01%)

rustbot assigned marcoieni Feb 11, 2025

jieyouxu reviewed Feb 11, 2025

View reviewed changes